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ABSTRACT 

The second year of operation of the Transition 
Program for Refugee Children (T.P.R.C.) is evaluated in this report. 
The program provides classes in English as a Second Language (E.S.L.) 
or content area instruction with an E.S.L. approach specifically 
designed to improve students' English language listening, speaking, 
reading, and writing abilities. The program also attempts to 
acclimatize students to American life and culture. Available data 
suggests that T.P.R.C. students are progressing in their knowledge of 
English syntax and mathematics. Generalizations about progress of the 
whole group are tentative, due to problems experienced in collecting 
information. Difficulties were aggravated further by the late receipt 
of funds. A further problem was that of identifying refugee students. 
The Immigration and Naturalization Service decides who should be 
awarded refugee status, but there is no accepted definition of 
refugee. ( icDN ) 
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1983-1984, second year of funding 



Central Administration: 
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Brooklyn, New ftrrk 



Number of Participants: 



813 students in grades 9 to 12 



Participating Sit 



15 high schools 1n New York City 
(13 public, two private) . 



Contact Person: 



E11 Plotkin, Program Manager 
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I. OVERVIEW 



In its second year of operation, the Transition Program for 
Refugee Children (T.P.R.C.) continued to provide classes in English as 
a second language (E.S.L.) or content-area instruction with an E.S.L. 
approach specifically designed to Improve participating students' 
English language listening, speaking, reading, and writing abilities. 
The program also attempted to acclimate students to American life and 
cul ture. 

Federal funding under the Refugee Act for New York City high 
schools, which totaled $126,958, was provided on a per capita formula based 
on the number of New York City students identified as refugees. However, 
as the federal definition of refugee 1s narrow, the actual number of 
refugee students served by the program is greater than the number on 
which funding was based. By federal guidelines, school personnel' reported 



( 



313 refugees in the 15 schools with the largest populations. Funds were 
distributed according to the original estimate of the refugee population 

as follows: ^ 

--18 thousand dollars in instructional materials' were dis- 
tributed per capita to the 26 public and non-public schools 
that had five or more refugee students; 

--one non-public school was assigned a paraprofessional ; 

--■six teaching positions were split among the 13 public 
schools with the largest refugee populations. 

Of the 13 public schools provided with teaching positions, six 
offered one class each, four offered two classes each, two offered three 
classes each, and one offered ten classes. 

Given the limited nature of the funding, Refugee Act monies 
were necessarily combined with funds from other sources in the individual 
schools. As a result, the services provided to an entitled student 
varied considerably from site to site. The necessary mixing of funds, 
variations in services, and inconsistent receipt of funding, made it 
impossible to isolate the effect of T.P.R.C. funding for evaluation 
purposes. 

Two schools, Theodore Roosevelt and FranklinD. Roosevelt, had 
the largest populations of refugee students. Their combined total of 
293 represents 36 percent of the total number of students reported. 
Table 1 presents the number of program students by site, and Table 2 
presents a breakdown of the students by grade. 

Program students were fairly evenly distributed among grades 
nine through eleven, which had over 200 students each. Few program 
students (64) were in the twelfth grade. 
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TABLE 1 

Number of Program Students by Site 



School 



Number of Students 



Seward Park 
Martin L. King 
Park West 
C. Columbus 
Walton 

T. Roosevelt 

Midwood 

New Utrecnt 

Fort Hamilton 

Franklin D. Roosevelt 

Long Island City 

Newtown 

Grover Cleveland 
Solomon Si hechter 
Free School 

TOTAL 
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41 
60 
59 
58 
61 

157 
26 
21 
23 

136 
46 
34 
■35 
12 
44 



813 



TABLE 2 



Number of Program Students by Grade ^ 



Grade 



Number of Students 



9 


264 


10 


273 


11 


212 


12 


64 


TOTAL 


813 
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II. FINDINGS- 

a * » 

This section presents the assessment Instruments and procedures, 
and the test .results used to evaluate student achievement in 1983-84. The 
data set for the Transition Program for Refugee Children was assembled ' 
by matching a Division of High Schools roster of participating students 
to the Office of Educational A&sessment high school data base containing 
Criterion Referenced English Syntax Test (CREST) scores and the New York 
City Reading Test and New York City Mathematics Test scores. Of the 813 
st.udents in the program, complete CREST data were available for 240 
students in the fall and 228 in the spring. To provide an alternative 

4 

indicator of English achievement for students for whom CREST data were 
unavailable, a match was done to seek New York City Reading Test scores. 
However, students who were missing CREST scores did not have New York 
City Reading Test scores either. Scores on the New York City Mathematics 
Test were available for 160 students. 

The number of cases with complete test data identified by the 
computer matches was low. Errors in data entry generally account 
for some loss of information, especially when files are matched, but 
test scores may not have been reported for many program students. The 
possible reasons for this are many: some students may have been absent 
for testing; others may not have taken the city-wide reading and math 
pre-tests because they entered school late in the year; and many would 
have been excused from the city-wide tests due to their recent immigrant 
status. Because attrition may not have been an entirely random process, 
the resulting subset of students for whom complete data are available 
may not be representative of the program population. 



ACQUISITION OF ENGLISH SYNTAX 

The CREST, used to measure English language achievement, was 
developed by the New York City Public Schools to measure mastery of 
the E.S.L. curriculum objectives at tne high school level. It is the 
instrument used to assess English language acquisition in Chapter I E.S.L. 
programs in New York City. There are four items for each objective,, 
and mastery of an objective is achieved when three these items are 
answered correctly. The test has three levels: beginning (I), inter- 
mediate (II), and advanced (III). The maximum score on Levels I and II 
is 25, while the maximum score on Level III is 15. 

In the absence of a criterion set as program objective, the 
city-wide objective for Chapter I E.S.L. students (mastery of one CREST 
objective for every foyr weeks of school attendance) was used to judge 
program outcomes. The CREST was administered to T.P.R.C. students at 
the beginning and end of each semester. Mean differences between pre- 
test and post-test were calculated to represent the gain score. Since 
the test's levels are not equated vertically, it was impossible to 
measure gains for students who changed levels. Extensive information 
on CREST objectives and psychometric properties appear in the Technical 
' Manual, New York City English as a Second Language Criterion Referenced 

Engl i sh Syntax Test . 

Examination of Table 3 reveals that, overall, students generally 
met the objective in both fall (1.46 objectives per month) and spring 
(1.18 objectives per month). Students tested on Levels I and II mastered 

*Board of Education of the City of New York, Division of High Schools, 1978. 
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over 1.6 objectives per jmonth of attendance in the fall term. Those 
students tested on these same levels 1n the spring mastered an average 
of 1.4 objectives per iifonth. Students tested on Level III gained ap- 
proximately 0.8 objectives per month of attendance du/ing both fall and 
spring terms. 

TABLE 3 

Results of the Criterion Referenced English Syntax Test 
(T.P.R.C. Students Pre- and Post-Tested on Same Test Leve 1 . ) 



" — " Average 

Average Number of Averag Average Objectives 

Test Number of Objectives Mastered: Objectives Months of Mastered 

Level Students Pre Post Mastered* Treatment Per Month 



Fall 



I 


79 


10.30 


15.88 


5.58 


3.30 


1.69 


II 


105 


11.96 


17.20 


5.24 


3.21 


1.63 


III 


56 


9.25 


11.91 


2.66 


3.27 


0.81 


TOTAL 


240 


10.78 


15.53 


4.75 


o.25 


1.46 








Spring 








I 


63 


11.05 


15.52 


4.47 


3.22 


1.39 


II 


86 


14.50 


18.87 


4.37 


3.20 


1.40 


III 


79 


9.28 


11.76 


2.46 


3.19 


0.77 


TOTAL 


228 


11.74 


15.48 


3.74 


3.20 


1.18 


*Post 


-test minus 


pre-test. 


f 
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Students tested on Levels I'and II, therefore, exceeded dty- 
wlde Chapter I guidelines which dfli for a gain of one CREST objective 
for every month of attendance. Level III students approached but did 
not meet these guialjines^ partly because it takes longer to master an 
objective at that level. The Chapter I objective assumes that progress 
in the the acquisition of English syntactic skills 1s linear; the achieve 
ment of LeveJ^ III students suggests that it is not. 

« 

MATHEMATICS ACHIEVEMENT 

Mathematics achievement was measured with the New York City 
Mathematics Test , a New York-normed version of the Stanford Test of 
Academic Skills (TASK), which is designed to measure general mathematics 
competence. *It emphasizes arithmetic and numeric concepts and applica- 
tions with minor emphasis on algebra, geometry, and measurement. The 
TASK, which has two forms, is a two-level test. Level I is designed for 

grades eight, nine, and ten; Level II for grades eleven and twelve and 

* 

junior college level. The TASK was normed on two national samples. 

Effect size was calculated for each grade level following the 

* it ; 

procedure recommended by Cohen. An effect ^size for correlated t-test 
is a ratio of the average gain to the standard deviation of the gains. 
Unlike the t-test, it is freed of sample size. Effect size provides 



*Further information is available from the Psychological Corporation, 
7500 Old Oak Blvd., Cleveland, Ohio 44130. 

*J. Cohen: Statistical Power Analysis for the Behavioral Sciences (Revised 
Edition). New York: Academic Press, 1977, Chapter 2 
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^'additional substance to the analysis as it may be interpreted in light 
of Cohen's recommendations: 

.20 = small effect size 

.50 » moderate effect size 

.80 - large effect size 
Results of this test are presented in Table 4. Refugee Program 
students 1 manifested statistically significant- raw score gains in grades 
ten and eleven. Effect sizes for these grades were moderate. Results 
for a small group of ninth-grr.Je students did not reach statistical 
significance and the accompanying effect size was correspondingly small. 

TABLE 4 

^ Mathematics Achievement 

Significance of the Total Raw Score Differences Between 
Initial and Final Test Scores of Students on the 
New York City Mathematics' Test , by Grade 



Pre-Test Post-Test 

Standard Standard Mean Effect 

Grade N Mean Deviation Mean Deviation Difference Size 



9 


33 


26.4 


10.5 


27.8 


9.6. 


1.4 


.18 


10 


62 


28.0 


10.0 


30.7 


9.2 


** 

2.7 


.42 


11 


65 


21.8 


9.7 


25.2 


9.2 


** 

3.5 


.54 



**Si gnifi cant V the .01 level. 
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III. CONCLUSIONS AND RECOMMENDATIONS 

The available data suggest that, in the second year of funding, 
T.P.R.C. students are progressing in their knowledge of English syntax 
and in mathematics. Nevertheless, although the program students met New 
York City guidelines for, English language achievement, complete data 
concerning the CREST and New York City Mathematics Test are not available 
for enough students to accurately evaluate achievement in these areas. 

The sources of the difficulty in collecting information orv 
T.P.R.C. students city-wide are many, and have been indicated in the 
introductory section of this report. The number of students with 
missing or incomplete scores, however, suggests that efforts should 
continue to be made in the participating schools to test all students 
and report the information as completely as possible. 

Clearly, the limited number of cases reported indicates that 
generalizations about the progress of the whole gfoup of students should 
be made with caution, at best. In addition, the combination of funds to 
create coherent academic programs, though it is educationally correct 
and sensible, makes the attribution of growth to an individual funding 
source questionable. A better approach to the evaluation of the progress 
made by T.P.R.C. students would be to conduct a global assessment of the 
delivery of services to LEP students across funding sources. 

Another issue involves the implementation of services supported 
by the Refugee Act. Funding for the 1983-84 academic year was received 
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after the school ^ear had begun, making it difficult to plan for the 
optimal use of the funds. This might have resulted, for example, in 
programs not being able to identify and place staff in the fall, possibly 
resulting in less systematic or extensive services during the year due 
to the lack of personnel. Clearly, timely provision of funding would 
improve the planning and implementation of services to T.P.R.C. students. 

A final issue is the identification of refugee students. The 
Immigration and Naturalization Service decides who should be awarded 
refugee status, but there is no accepted definition of refugee. The 
result is that, while there technically may be refugees from allied 
countries such as Greece or Turkey, teachers do not think of them 
refugees. On the other hand, students from Haiti and Cuba, who i. . not 
be awarded refugee status, are often considered refugees by teachers 
because of newspaper references to these immigrants as "refugees." 
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